System and method for autonomic tuning the number of threads in application server at runtime

ABSTRACT

An autonomic performance tuning system with a fuzzy control model is provided for autonomic tuning the number of threads in application server at runtime. It dynamically monitors and measures behaviours of an application server, feeds these measures into the fuzzy control model and uses the results derived from the fuzzy control model to adaptively tune the number of threads in the application server. This process is executed in a feedback-loop manner to provide optimal performance at all time. It frees administrators from manually tuning the number of threads and enables application server self-tuning and self-optimization.

CLAIM OF PRIORITY

This application claims priority from provisional application entitledMETHOD FOR ADJUSTING THE NUMBER OF THREADS DYNAMICALLY IN J2EEAPPLICATION SERVER SYSTEMS, Application No. 60/821,358, filed on Aug. 3,2006, by Yan Zhang, and incorporated herein by reference.

FIELD OF THE INVENTION

The invention is related generally to application servers, andspecifically to a system and method for tuning the number of threads inapplication server at runtime to optimise the performance of applicationserver. The system and method can be further tailored to suit tuningother parameters in application server and to meet other system qualityrequirements such as security.

BACKGROUND

In a typical e-business environment, application server resides in themiddle tier of the multi-tiered e-business system architecture. Its roleis to provide middleware services (for example, transactions, securityand persistence) to deployed applications. FIG. 1 shows a typicaltopology of an application server deployment. In a Web application, anInternet browser 102 submits HTTP (hypertext transfer protocol) 104requests to and downloads HTML (hypertext markup language) pages from aWeb Server 106. The Web Server 106 directs requests to an applicationserver 108. The application server 108 is connected to a back-enddatabase 110 where persisted data are stored. In an application notdeployed using a browser, stand-alone Java clients or applets 112 can beused, and these Java clients or applets 112 would communicate directlywith the application server 108, using the Java Remote Method Invocation(RMI) 114 as the underlying protocol.

The performance of application server has important effect on theperformance of the whole e-business system. Experience has shown thatthe actual number of threads in application server used to serve therequests from clients has a significant impact on the performance ofapplication server. Therefore, it is critical to tune the number ofthreads in application server. Current application servers have providedthread pool size parameter for administrator to manually adjust thenumber of threads in application server. However, figuring out theoptimal thread pool size (number of threads) is not an easy task.

In practice, administrators experimentally discover the thread pool sizethat provide the desired levels of performance via performance testingusing some industry benchmarks or in-house applications given certainworkloads. However, the experimental approach is a time-consuming,expensive and non-trivial method. It requires highly skilled personnel.In addition, since the workloads of e-business sites tend to varydynamically and exhibit fluctuations, even if the thread pool size istuned well at one point in time, it will show poor performance at othertimes. That is, a statically tuned system will perform poorly in thetime-varying workloads as present in typical e-business environment.

Therefore, what is needed is a means for automatically tuning the numberof threads in application server at runtime to achieve optimal systemperformance at all time.

SUMMARY

An objective of the present inventions is to optimise the performance ofapplication server.

A further objective of the present invention is to eliminate manualtuning effort and to enable application server self-tuning andself-optimization.

In accordance with an embodiment of the invention, a system and methodfor autonomic tuning the number of threads in application server atruntime is provided. In accordance with an embodiment, an autonomicperformance tuning system is built to perform monitoring, decision andtuning operations on application server without human intervention. Afuzzy control model is constructed and used in the autonomic performancetuning system. A fuzzy inference engine in the fuzzy control model makesdecisions on whether the number of threads needs to be adjusted or notand how many threads should be increased or decreased based on the fuzzycontrol algorithm and fuzzy rules in the rule base in the fuzzy controlmodel.

In accordance with an embodiment of the invention, during runtime, thesystem collects the response time of clients' requests and the number ofthreads handing the requests in application server; the collectedresponse time and the number of threads are fed into the fuzzy controlmodel; the fuzzy inference engine in the fuzzy control modelsubsequently evaluates activation strength of the available fuzzy rulesin the rule base, combines their rule-consequent sides and determineswhether the number of threads needs to be adjusted or not and how manythreads should be increased or decreased; the results derived from thefuzzy inference engine are translated into actions on the runningapplication server. This process is executed in a feedback-loop mannerto provide optimal performance at all time.

The overall result of the system and method is a dramatic performanceimprovement of application server. The system and method can be furthertailored to suit tuning other parameters in application server and tomeet other system quality requirements such as security.

BRIEF DESCRIPTION OF THE FIGURES

FIG. 1 shows an illustration of a typical topology of an applicationserver deployment.

FIG. 2 shows an illustration of the structure of the autonomicperformance tuning system with the fuzzy control model.

FIG. 3 shows an illustration of determining the set of fuzzy rules.

FIG. 4 shows an illustration of membership function for input variables.

FIG. 5 shows an illustration of membership function for outputvariables.

FIG. 6 shows an illustration of the fuzzy control algorithm.

DETAILED DESCRIPTION

Currently, application server plays an important role in a typicale-business environment. It resides in the middle tier of themulti-tiered e-business system architecture and provides the middlewareservices, such as transactions, security and persistence, to thedeployed applications. Experience has shown that the actual number ofthreads used in the application server has significant effect on thesystem performance. Application servers provide the thread pool sizeparameter for administrators to manually adjust the number of threads.However, manual tuning the thread pool size is time consuming andrequires highly skilled personnel. In addition, since the workloads ofe-business sites tend to vary dynamically and exhibit fluctuations, evenif the thread pool size is adjusted well at one point in time, it willshow poor performance at other times.

In accordance with an embodiment of the present invention, an autonomicperformance tuning system automatically adjusts the number of threads inapplication server at runtime to achieve optimal response time. It alsomeets objectives of eliminating manual tuning effort and enablingapplication server self-tuning and self-optimization.

In accordance with an embodiment of the invention, a system and methodfor autonomic tuning the number of threads in application server atruntime is provided. In accordance with an embodiment, an autonomicperformance tuning system is built to perform monitoring, decision andtuning operations on application server without human intervention. Afuzzy control model is constructed and used in the autonomic performancetuning system. A fuzzy inference engine in the fuzzy control model makesdecisions on whether the number of threads needs to be adjusted or notand how many threads should be increased or decreased based on the fuzzycontrol algorithm and fuzzy rules in the rule base in the fuzzy controlmodel.

In accordance with an embodiment of the invention, during runtime, thesystem collects the response time of clients' requests and the number ofthreads handing the requests in application server; the collectedresponse time and the number of threads are fed into the fuzzy controlmodel; the fuzzy inference engine in the fuzzy control modelsubsequently evaluates activation strength of the available fuzzy rulesin the rule base, combines their rule-consequent sides and determineswhether the number of threads needs to be adjusted or not and how manythreads should be increased or decreased; the results derived from thefuzzy inference engine are translated into actions on the runningapplication server. This process is executed in a feedback-loop mannerto provide optimal performance at all time.

The overall result of the system and method is a dramatic performanceimprovement of application server. The system and method can be furthertailored to suit tuning other parameters in application server and tomeet other system quality requirements such as security.

FIG. 2 shows an illustration of the structure of the autonomicperformance tuning system with the fuzzy control model, in accordancewith an embodiment of the invention. As shown in FIG. 2, thefunctionalities of the system are grouped into three main logicalmodules, namely the performance monitor module 202, performancecontroller module 204 and regulator module 206. The three functionalmodules constitute an automated, feedback loop. The running environmentof the application server 200 is monitored. Which tuning strategy ischosen is decided by the performance controller module 204. Theperformance of the application server 200 is optimised and adapted tomeet its high-level performance goals in the current environment. Theresulting system is subsequently monitored and evaluated, optimizationand tuning strategies being possibly tuned in effect.

In accordance with an embodiment of the invention, the performancemonitor module 202 consists of Data Collector component 208, DataProcessor component 210, and a database 212. The Data Collector 208obtains observations of real-time performance data. Example performancedata to collect include request arrival rate and response time etc. Thecollected data are processed by Data Processor 210 to measurements whichare stored in a database 212 so that they can be reused or to meet thestatistical requirements.

In accordance with an embodiment of the invention, the regulator module206 consists of the tuner component 230. The tuner 230 translates thestrategy chosen by the Performance Controller Module 204 into actions,and injects the actions into the application server 200 to meet thesystem performance requirements.

In accordance with an embodiment of the invention, the performancecontroller module 204 consists of the comparator component 216 and thefuzzy control model 232. The performance controller module 204 isresponsible for choosing the right performance tuning strategy for thesituation signalled by the monitoring module 202 and Service LevelAgreement (SLA) 214. In accordance with an embodiment of the invention,the SLA 214 specifies response time optimization.

In accordance with an embodiment of the invention, the comparator 216compares the difference between the real-time performance data such asresponse time and the level of service specified by the agreement. Inaccordance with an embodiment of the invention, the comparator 216calculates the changes in the number of threads and response time. Thecalculated results are feed into the fuzzy control model 232.

In accordance with an embodiment of the invention, the fuzzy controlmodel 232 consist of the six components:

1. Input normalization 218: Perform a scale transformation which mapsthe actual real-world values of current variables from the comparatorcomponent into a normalized universe of discourse.

2. Fuzzification 220: Calculate fuzzy input, that is, convert a crispvalue (precise numerical value) into a fuzzy set to make it compatiblewith the fuzzy set representation of the linguistic variable in therule-antecedent.

3. Rule Base 222: This includes linguistic fuzzy rules and membershipfunctions of the linguistic values.

Fuzzy Rules

The fuzzy rules in the rule base take the form: IF [conditions] THEN[actions], where conditions and actions are linguistic labels applied toinput and out variables respectively. In accordance with an embodimentof the invention, the difference in the number of threads between thecurrent and previous time intervals and the difference in the responsetime between the current and previous time intervals are inputvariables, and the number of threads to be changed in next time intervalis output variable.

The fuzzy rules are determined based on the experience that the numberof threads has the concave upward effect on response time shown as inFIG. 3. To illustrate how to determine the set of fuzzy rules, supposethat the current number of threads is on the right of the optimal valueshown as at point A and point B in FIG. 3; we would like to decrease thenumber of threads. However, since the optimal value is not known, thefuzzy rules should first determine whether it is to the left or to theright. The “IF” part of the fuzzy rules determines the position on theresponse time curve and the distance from the optimal point. Forexample, if we increase the number of threads and the response time alsoshows a large increase, then we are far to the right of the optimalvalue. The “THEN” part indicates the suggested action, that is, tochange the number of threads in the opposite direction (decreasing thenumber of threads by a large amount). Conversely, if we are to the leftof the optimal value at point C and point D, we would like to increasethe number of threads. Since we do not know the actual the number ofthreads that minimize the response time, these rules are described interms of difference in the number of threads and difference in theresponse time values between the current and previous time intervals.

Two examples of the fuzzy rules are as follows:

If the difference in the number of threads between the current andprevious time intervals is Small Positive and the difference in theresponse time between the current and previous time intervals is SmallPositive, then the number of threads to be changed in next time intervalis Small Negative.

If the difference in the number of threads between the current andprevious time intervals is Small Positive and the difference in theresponse time between the current and previous time intervals is LargePositive, then the number of threads to be changed in next time intervalis Large Negative.

Membership Functions

The linguistic values of a fuzzy variable can use Gaussian, triangularor trapezoidal shaped membership functions.

In accordance with an embodiment of the invention, triangular membershipfunctions are used. FIG. 4 shows an illustration of membership functionfor input variables (the difference in the number of threads between thecurrent and previous time intervals and the difference in the responsetime between the current and previous time intervals). FIG. 5 shows anillustration of membership function for output variable (the number ofthreads to be changed).

In FIG. 4 and FIG. 5, LN represents Large Negative, MN represents MediumNegative, SN represents Small Negative, Z represents Zero, SP representsSmall Positive, MP represents Medium Positive, and LP represents LargePositive, in accordance with an embodiment of the invention.

Other membership functions, for example, Gaussian and trapezoidal shapedmembership functions can also be used instead of triangular function.

4. Fuzzy inference Engine 224: Calculate fuzzy output, that is, take thefuzzy sets as input and produce output in the form of fuzzy sets via theevaluating activation strength of every rule and combining theirrule-consequent sides. This implements specific fuzzy control algorithmsbased on a set of linguistic fuzzy rules and a set of membershipfunctions for linguistic values defined in the rule base.

5. Defuzzication 226: Calculate actual output, that is, convert fuzzyoutput into a crisp value. In accordance with an embodiment of theinvention, the centre of gravity method (COG) is used asdefuzzification. Other methods of defuzzification, for example, thesingleton method, the mean of maximum method, and the weighted summethod, can also be used instead of COG.

COG divides the integral of the membership function of the output fuzzyset into half, and the defuzzified value y out marks the dividing point.Formally, in the continuous case this results in

$y_{out} = \frac{\int{y\; {\mu_{B^{*}}(y)}{y}}}{\int{{\mu_{B^{*}}(y)}{y}}}$

(μ_(B*) is the membership function of the output fuzzy set B*)

6. Output Denormalization 228: Map the crisp value of the control outputinto its real world domain.

In accordance with an embodiment of the invention, FIG. 6 shows thealgorithm used to compute the number of threads to be changed in nexttime interval given two inputs: the difference in the number of threads(x1) between the current and previous time intervals and the differencein the response time (x2) between the current and previous timeintervals. Step 602 obtains x1 and x2 values. Step 604 finds the valuesof all membership functions given the values for x1 and x2. Step 606finds the values for the premise membership functions for given x1 andx2 using the minimum operation. Step 608 finds the areas under themembership functions for all possible implied fuzzy sets. Step 610initializes the COG numerator and denominator values. Step 612 cyclesthrough all areas to determine COG. Step 614 outputs the value of thenumber of threads to be changed. After step 614, go back to step 602 andcontinue to compute the next number of threads to be changed given thenext two inputs.

The foregoing description of the present invention has been provided forthe purposes of illustration and description. It is not intended to beexhaustive or to limit the invention to the precise forms disclosed. Theembodiments were chosen and described in order to best explain theprinciples of the invention and its practical application, therebyenabling others skilled in the art to understand the invention forvarious embodiments and with various modifications that are suited tothe particular use contemplated. It is intended that the scope of theinvention be defined by the following claims and their equivalence.

What is claimed is:
 1. A system for autonomic tuning the number ofthreads in an application server at runtime, comprising: a performancemonitor to collect the response time of clients' requests and the numberof threads handing the requests in the application server; a fuzzyperformance controller to determine how to adjust the number of threadsin the application server based on a fuzzy control algorithm and fuzzyrules; A regulator to translate the decision made by the fuzzyperformance controller into actions and injects the actions into theapplication server.
 2. The system of claim 1 wherein said action may beincreasing the number of the threads, decreasing the number of thethreads or not changing the number of threads in the application server.3. The system of claim 1 wherein said fuzzy control algorithm consistsof the steps of: a. obtaining the values of the difference in the numberof threads and the difference in the response time between the currentand previous time intervals; b. finding the values of all membershipfunctions given the values of the difference in the number of threadsand the difference in the response time between the current and previoustime intervals; c. finding the values for the premise membershipfunctions given the values of the difference in the number of threadsand the difference in the response time between the current and previoustime intervals; d. finding the areas under the membership functions forall possible implied fuzzy sets; e. calculating the number of threadsneeded to be increased or decreased in next time interval by usingdefuzzification method.
 4. The system of claim 3 wherein membershipfunctions can be Gaussian, triangular or trapezoidal shaped membershipfunctions.
 5. The system of claim 3 wherein defuzzification method canbe the centre of gravity method (COG), the singleton method, the mean ofmaximum method, or the weighted sum method.
 6. The system of claim 1wherein said fuzzy rules are based on a priori knowledge.
 7. A methodfor autonomic tuning the number of threads in an application server atruntime, comprising the steps of: gathering the response time ofclients' requests and the number of threads handing the requests in theapplication server; calculating the difference in the response time andthe difference in the number of threads between the current and previoustime intervals; determining the next action based on a fuzzy controlalgorithm and fuzzy rules stored in fuzzy rule base; and injecting theactions into the application server.
 8. The method of claim 7 whereinsaid action may be increasing the number of the threads, decreasing thenumber of the threads or not changing the number of threads in theapplication server.
 9. The method of claim 7 wherein said fuzzy controlalgorithm consists of the steps of: a. obtaining the values of thedifference in the number of threads and the difference in the responsetime between the current and previous time intervals; b. finding thevalues of all membership functions given the values of the difference inthe number of threads and the difference in the response time betweenthe current and previous time intervals; c. finding the values for thepremise membership functions given the values of the difference in thenumber of threads and the difference in the response time between thecurrent and previous time intervals; d. finding the areas under themembership functions for all possible implied fuzzy sets; e. calculatingthe number of threads needed to be increased or decreased in next timeinterval by using defuzzification method.
 10. The method of claim 9wherein membership functions can be Gaussian, triangular or trapezoidalshaped membership functions.
 11. The method of claim 9 whereindefuzzification method can be the centre of gravity method (COG), thesingleton method, the mean of maximum method, or the weighted summethod.
 12. The method of claim 7 wherein said fuzzy rules are based ona priori knowledge.
 13. A computer readable medium, includinginstructions thereon which when executed cause the computer to performthe steps of: gathering the response time of clients' requests and thenumber of threads in an application server; calculating the differencein the response time and the difference in the number of threads betweenthe current and previous time intervals; determining the next actionbased on an fuzzy control algorithm and fuzzy rules stored in fuzzy rulebase; and injecting the actions into the application server.
 14. Thecomputer readable medium of claim 13 wherein said action may beincreasing the number of the threads, decreasing the number of thethreads or not changing the number of threads in the application server.15. The computer readable medium of claim 13 wherein said fuzzy controlalgorithm consists of the steps of: a. obtaining the values of thedifference in the number of threads and the difference in the responsetime between the current and previous time intervals; b. finding thevalues of all membership functions given the values of the difference inthe number of threads and the difference in the response time betweenthe current and previous time intervals; c. finding the values for thepremise membership functions given the values of the difference in thenumber of threads and the difference in the response time between thecurrent and previous time intervals; d. finding the areas under themembership functions for all possible implied fuzzy sets; e. calculatingthe number of threads needed to be increased or decreased in next timeinterval by using defuzzification method.
 16. The computer readablemedium of claim 15 wherein membership functions can be Gaussian,triangular or trapezoidal shaped membership functions.
 17. The computerreadable medium of claim 15 wherein defuzzification method can be thecentre of gravity method (COG), the singleton method, the mean ofmaximum method, or the weighted sum method.
 18. The computer readablemedium of claim 13 wherein said fuzzy rules are based on a prioriknowledge.